ostree.git
6 years agoci/rpmostree: Bump to 2019.3
Dan Nicholson [Fri, 26 Jul 2019 20:28:47 +0000 (14:28 -0600)]
ci/rpmostree: Bump to 2019.3

The test-ucontainer.sh test in rpmostree 2019.1 is failing for
non-ostree reasons. This is fixed in [1], which is part of 2019.3. I
believe this is still a relevant test of f29 since that's what's shipped
in f29-updates.

1. https://github.com/projectatomic/rpm-ostree/commit/45b6186be0826889d805ba55bee3be75b042ed35

Closes: #1892
Approved by: jlebon

6 years agolib/kargs: Don't expose OstreeKernelArgs autoptr cleanup on old glib
Dan Nicholson [Fri, 26 Jul 2019 16:42:42 +0000 (10:42 -0600)]
lib/kargs: Don't expose OstreeKernelArgs autoptr cleanup on old glib

Move the OstreeKernelArgs autoptr cleanup definition to
ostree-autocleanups.h, which will only expose the definitions when
building ostree or if glib is new enough. The include of
ostree-kernel-args.h needs to be moved before ostree-autocleanups.h in
ostree.h so that the OstreeKernelArgs type is declared when the autoptr
cleanup is defined. All the places it's used already pull in libglnx.h
first so that the compat macros are picked up if glib it too old during
the ostree build.

Closes: #1892
Approved by: jlebon

6 years agotests/gpg: Skip tests when subkeys can't be expired
Dan Nicholson [Fri, 26 Jul 2019 16:11:00 +0000 (10:11 -0600)]
tests/gpg: Skip tests when subkeys can't be expired

The ability to expire subkeys using gpg's --quick-set-expire is only
available on gnupg 2.1.22. If expiring a subkey fails, assume this is
why and skip the tests that require it but run the actions that the
subsequent tests depend on. This was failing on the Debian Stretch CI
tests since stretch has gnupg 2.1.18.

Closes: #1892
Approved by: jlebon

6 years agotests/gpg: Use exit hook to kill agent in temporary GPG homedir
Dan Nicholson [Fri, 26 Jul 2019 16:17:07 +0000 (10:17 -0600)]
tests/gpg: Use exit hook to kill agent in temporary GPG homedir

This wasn't available when I originally wrote this, but it ensures that
the running gpg-agent in tmpgpghome is killed in case the tests exit
early.

Closes: #1892
Approved by: jlebon

6 years agotests/sizes.js: Fix byte array unpacking
Dan Nicholson [Mon, 8 Jul 2019 21:08:07 +0000 (15:08 -0600)]
tests/sizes.js: Fix byte array unpacking

Recent GJS changed how byte arrays are unpacked with some assumptions
that they are likely strings. Manually use get_child_value() and
get_byte() to ensure the correct value is parsed when checking the
`ostree.sizes` metadata.

The upstream test is currently passing fine with GJS 1.56.2, but at
Endless we (unfortunately) have a downstream change that adds the object
type as an additional byte in the array. This is parsed incorrectly by
`deep_unpack()`. We can carry this patch downstream, but this change
makes the test more robust regardless.

Closes: #1884
Approved by: cgwalters

6 years agotests/sizes: Fix call to commit_transaction()
Dan Nicholson [Mon, 8 Jul 2019 21:39:11 +0000 (15:39 -0600)]
tests/sizes: Fix call to commit_transaction()

The GIR for commit_transaction() only has a single argument for the
GCancellable. Calling it with 2 arguments prints a GJS warning:

Gjs-Message: 15:37:40.287: JS WARNING: [/home/dan/src/ostree/tests/test-sizes.js 56]: Too many arguments to method OSTree.Repo.commit_transaction: expected 1, got 2

Currently this is harmless, but it could become a hard error in GJS at
some point.

Closes: #1884
Approved by: cgwalters

6 years agolib/kargs: add missing function to symbol-versioning table
Rafael Fonseca [Tue, 2 Jul 2019 11:26:25 +0000 (13:26 +0200)]
lib/kargs: add missing function to symbol-versioning table

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
Closes: #1883
Approved by: jlebon

6 years agolib/repo: Improve docs for get_min_free_space_bytes()
Matthew Leeds [Tue, 25 Jun 2019 23:58:38 +0000 (16:58 -0700)]
lib/repo: Improve docs for get_min_free_space_bytes()

Closes: #1879
Approved by: jlebon

6 years agolib/util: Drop unneeded ot-tool-util.h includes
Robert Fairley [Fri, 17 May 2019 18:23:41 +0000 (14:23 -0400)]
lib/util: Drop unneeded ot-tool-util.h includes

With `ot-tool-util.h` made visible in `otutil.h` (in
be2572bf68090a5e277338d2613d3c7d53b0c9e8), drop previous includes
of `ot-tool-util.h` elsewhere.

Closes: #1876
Approved by: zonggen

6 years agoci/flatpak: Bump FLATPAK_TAG to 1.4.1
Robert Fairley [Wed, 19 Jun 2019 22:22:11 +0000 (18:22 -0400)]
ci/flatpak: Bump FLATPAK_TAG to 1.4.1

Bump the flatpak version used in CI to avoid a repeated definition of
`renameat2()` in libglnx and glibc.

See: https://github.com/ostreedev/ostree/pull/1871#issuecomment-503768062

Closes: #1871
Approved by: jlebon

6 years agoMakefile-libostree.am: Work around clang issues with g-ir-scanner
Robert Fairley [Wed, 19 Jun 2019 17:32:00 +0000 (13:32 -0400)]
Makefile-libostree.am: Work around clang issues with g-ir-scanner

Work around an issue with clang-7 in Fedora 29 where the
`-fstack-clash-protection` flag is not recognized. Force
the compiler used by `g-ir-scanner` to `gcc`, so that the flag
is recognized.

Related: https://github.com/projectatomic/rpm-ostree/pull/1787#issuecomment-473971585

Closes: #1871
Approved by: jlebon

6 years agoci/build-check: Remove unused-command-line-argument diagnostic
Robert Fairley [Wed, 19 Jun 2019 20:04:39 +0000 (16:04 -0400)]
ci/build-check: Remove unused-command-line-argument diagnostic

Since `clang-4.0` is no longer being used, it should be safe to remove
use of this diagnostic for clang. This enables using `gcc` as a workaround
for `g-ir-scanner`, as `unused-command-line-argument` is not recognized by
`gcc`.

See: https://github.com/ostreedev/ostree/pull/1871#issuecomment-503765624

Closes: #1871
Approved by: jlebon

6 years agoci: Use Fedora 29 artifacts
Robert Fairley [Tue, 11 Jun 2019 20:38:16 +0000 (16:38 -0400)]
ci: Use Fedora 29 artifacts

Use Fedora 29 artifacts instead of Fedora 28, since 28 is now
end-of-life.

Also rename `ci/fah28-insttests.sh` -> `ci/fah29-insttests.sh`
and use the https://getfedora.org/atomic_qcow2_latest redirect
URL for the latest Fedora Atomic Host 29 image.

Closes: #1871
Approved by: jlebon

6 years agotests: Always cleanup gpg-agent when exiting
Dan Nicholson [Thu, 13 Jun 2019 21:26:47 +0000 (16:26 -0500)]
tests: Always cleanup gpg-agent when exiting

Add `libtest_cleanup_gpg()` to the array of commands to run when
exiting. This provides 2 improvements:

1. You don't need to worry about whether the test will spawn a gpg-agent
   and therefore require adding a call to `libtest_cleanup_gpg()`.

2. All the existing users were calling `libtest_cleanup_gpg()` at the
   end of the script. If there was a failure and the script exited
   early, then it wouldn't cleanup and there may be a stray gpg-agent
   hanging around.

Closes: #1799
Approved by: cgwalters

6 years agotests/libtest: Allow appending actions to be run on EXIT
Dan Nicholson [Thu, 13 Jun 2019 20:57:17 +0000 (15:57 -0500)]
tests/libtest: Allow appending actions to be run on EXIT

Currently if a test script adds a trap on `EXIT` to run some cleanup, it
will stomp on the existing trap to run `save_core()`. Allow for scripts
to append actions that will run on exit by introducing an array that
will be iterated over by a single exit runner.

Closes: #1799
Approved by: cgwalters

6 years agolib: Kill GPG agent when cleaning up tmp homedirs
Dan Nicholson [Thu, 10 Jan 2019 20:00:42 +0000 (14:00 -0600)]
lib: Kill GPG agent when cleaning up tmp homedirs

When a temporary directory is used for GPG operations, it's pretty clear
that the running agent will be useless after the directory is deleted.
Call the new `ot_gpgme_kill_agent ()` helper to kill gpg-agent rather
than leaving them it hanging around forever.

As it turns out, gnupg does have code to make gpg-agent automatically
exit when the homedir is removed (https://dev.gnupg.org/T2756), but
that's only available on gnupg 2.2 or newer. Possibly this code can be
dropped later when that's more widely deployed or users/distros have
been advised to backport the necessary changes.

Closes: #1799
Approved by: cgwalters

6 years agolib/gpg: Add helper to kill GPG agent
Dan Nicholson [Thu, 10 Jan 2019 19:49:17 +0000 (13:49 -0600)]
lib/gpg: Add helper to kill GPG agent

With GnuPG 2, any time you do basically any operation, a gpg-agent will
be spawned for the GPG home directory in use. The classic way to kill a
gpg-agent is to use `gpg-connect-agent` and send the `killagent` command
as is done in libtest.sh.

Closes: #1799
Approved by: cgwalters

6 years agotests/test-gpg-signed-commit: Test more key states
Dan Nicholson [Mon, 17 Jun 2019 20:25:22 +0000 (15:25 -0500)]
tests/test-gpg-signed-commit: Test more key states

Extend test-gpg-signed-commit.sh to test various key states. If gpg is
found that supports the required options, keys will be generated on the
fly and changed in various ways to exercise the output from
`ostree_gpg_verify_result_describe_variant` used in `ostree show`.

I tested this using gnupg 2.2.12, so I hope it works well enough on
various gpgs found in the wild.

Closes: #1872
Approved by: cgwalters

6 years agotests/libtest: Allow specifying GPG homedir to cleanup
Dan Nicholson [Mon, 17 Jun 2019 18:37:35 +0000 (13:37 -0500)]
tests/libtest: Allow specifying GPG homedir to cleanup

In case the tests want to use a custom GPG homedir, allow passing in the
homedir to use when cleaning up a running gpg-agent.

Closes: #1872
Approved by: cgwalters

6 years agolib/gpg: Show if GPG key has been revoked
Dan Nicholson [Mon, 10 Jun 2019 17:15:04 +0000 (12:15 -0500)]
lib/gpg: Show if GPG key has been revoked

Rather then showing the incorrect `BAD signature`.

Closes: #1872
Approved by: cgwalters

6 years agolib/gpg: Show information for expired keys
Dan Nicholson [Mon, 10 Jun 2019 17:05:21 +0000 (12:05 -0500)]
lib/gpg: Show information for expired keys

Introduce a new signature attribute for the key expiration timestamp and
display it when the key has a non-zero expiration time. Without this,
the error shown is `BAD signature`, which isn't correct.

Closes: #1872
Approved by: cgwalters

6 years agolib/gpg: Factor out expiration timestamp formatting
Dan Nicholson [Mon, 10 Jun 2019 17:01:25 +0000 (12:01 -0500)]
lib/gpg: Factor out expiration timestamp formatting

This will be reused to format a key expiration message.

Closes: #1872
Approved by: cgwalters

6 years agolib/kargs: Make API public and upstream new rpm-ostree APIs
Allen Bai [Tue, 28 May 2019 18:02:54 +0000 (14:02 -0400)]
lib/kargs: Make API public and upstream new rpm-ostree APIs

This change makes public the current kargs API in src/libostree/ostree-kernel-args.c
and adds documentations.

Upstreams the new kargs API from rpm-ostree/src/libpriv/rpmostree-kargs-process.c

Merges libostree_kernel_args_la_SOURCES to libostree_1_la_SOURCES in Makefile-libostree.am

Upstreams tests/check/test-kargs.c from rpm-ostree.

Closes: #1833
Closes: #1869
Approved by: jlebon

6 years agoadmin: allow multiple args for pin subcommand
Rafael Fonseca [Mon, 17 Jun 2019 14:41:00 +0000 (16:41 +0200)]
admin: allow multiple args for pin subcommand

Fixes #1816

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
Closes: #1874
Approved by: rfairley

6 years agobin/refs: Allow overwriting existing ref
Dan Nicholson [Mon, 10 Jun 2019 14:47:44 +0000 (09:47 -0500)]
bin/refs: Allow overwriting existing ref

Currently if you want to update a non-alias ref, you need to first check
if it exists and use either `ostree refs --create` or `ostree reset` as
appropriate. That's unnecessarily complicated and is much less
convenient than the old `write-refs` builtin that simply called
`ostree_repo_set_ref_immediate()` without any checks.

Add a `--force` option to be used with `--create` that does not raise an
error when the destination ref already exists.

Closes: #1870
Approved by: jlebon

6 years agolib/util: Fix segfault when validating filename
Allen Bai [Wed, 5 Jun 2019 14:30:31 +0000 (10:30 -0400)]
lib/util: Fix segfault when validating filename

This change fixes the segfault issue when calling ostree_repo_checkout_tree with
empty GFileInfo. A simple condition check for NULL value is added at
src/libotutil/ot-unix-utils.c:46. Closes: ostreedev#1864.

Closes: #1868
Approved by: jlebon

6 years agoostree-repo-finder: Clarify immutability after construction
Philip Withnall [Fri, 24 May 2019 12:43:34 +0000 (13:43 +0100)]
ostree-repo-finder: Clarify immutability after construction

I think this was the intention, but I forgot to document it.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #1863
Approved by: mwleeds

6 years agotests: Test committing archives from stdin
Tristan Van Berkom [Tue, 21 May 2019 13:30:03 +0000 (22:30 +0900)]
tests: Test committing archives from stdin

This updates test-libarchive.sh to test committing archives from
stdin and verifying their content is correct.

Closes: #1862
Approved by: jlebon

6 years agoman: Document ability to commit archives from stdin
Tristan Van Berkom [Tue, 21 May 2019 13:28:19 +0000 (22:28 +0900)]
man: Document ability to commit archives from stdin

Closes: #1862
Approved by: jlebon

6 years agobin/commit: Support committing archives from stdin
Tristan Van Berkom [Tue, 21 May 2019 13:26:55 +0000 (22:26 +0900)]
bin/commit: Support committing archives from stdin

This commit interprets archive filenames specified as '-' to
indicate that the input should be read from stdin.

Closes: #1862
Approved by: jlebon

6 years agolib/commit: Added new ostree_repo_write_archive_to_mtree_from_fd()
Tristan Van Berkom [Tue, 21 May 2019 13:01:09 +0000 (22:01 +0900)]
lib/commit: Added new ostree_repo_write_archive_to_mtree_from_fd()

Similar to ostree_repo_write_archive_to_mtree(), but takes
a file descriptor to read the archive from instead of mandating
a file path.

Usefull for importing archives into an OSTree repo over a socket
or from standard input in command line tools.

Closes: #1862
Approved by: jlebon

6 years agoci: Split out installdeps.sh from build.sh
Robert Fairley [Thu, 23 May 2019 17:05:07 +0000 (13:05 -0400)]
ci: Split out installdeps.sh from build.sh

This script is useful to run individually when setting up a
development environment for OSTree.

Closes: #1865
Approved by: jlebon

6 years agolib/fetcher-util: Use GIOErrorEnum instead of GIOError
Robert Fairley [Wed, 8 May 2019 23:15:53 +0000 (19:15 -0400)]
lib/fetcher-util: Use GIOErrorEnum instead of GIOError

Use GIOErrorEnum as the return value for
_ostree_fetcher_http_status_code_to_io_error(), to avoid an
implicit cast from GIOError.

Closes: #1857
Approved by: cgwalters

6 years agotests: Drop unused variable
Robert Fairley [Wed, 8 May 2019 23:13:12 +0000 (19:13 -0400)]
tests: Drop unused variable

Closes: #1857
Approved by: cgwalters

6 years agolib: add parameter annotations for sysroot_load_if_changed
Felix Krull [Sat, 18 May 2019 19:16:28 +0000 (21:16 +0200)]
lib: add parameter annotations for sysroot_load_if_changed

Closes: #1861
Approved by: cgwalters

6 years agolib: add some missing (out) annotations
Felix Krull [Sat, 18 May 2019 14:12:30 +0000 (16:12 +0200)]
lib: add some missing (out) annotations

Closes: #1861
Approved by: cgwalters

6 years agolib: fix up some nullability annotations
Felix Krull [Sat, 18 May 2019 11:58:34 +0000 (13:58 +0200)]
lib: fix up some nullability annotations

Closes: #1861
Approved by: cgwalters

6 years agolib: add minimum version to various symbols based on libostree-released.sym
Felix Krull [Fri, 17 May 2019 20:24:52 +0000 (22:24 +0200)]
lib: add minimum version to various symbols based on libostree-released.sym

Closes: #1861
Approved by: cgwalters

6 years agolib: fix doc comment for ostree_repo_get_min_free_space_bytes
Felix Krull [Fri, 17 May 2019 19:33:06 +0000 (21:33 +0200)]
lib: fix doc comment for ostree_repo_get_min_free_space_bytes

Closes: #1861
Approved by: cgwalters

6 years agoREADME: update link to ostree-rs
Felix Krull [Fri, 17 May 2019 15:35:12 +0000 (17:35 +0200)]
README: update link to ostree-rs

Closes: #1860
Approved by: jlebon

6 years agolib: mmap the summary to reduce RSS in clients
Richard Hughes [Wed, 8 May 2019 19:23:17 +0000 (20:23 +0100)]
lib: mmap the summary to reduce RSS in clients

Closes: #1855
Closes: #1856
Approved by: cgwalters

6 years agodocs: fix missing as typo
Tristan Cacqueray [Sun, 5 May 2019 03:31:52 +0000 (03:31 +0000)]
docs: fix missing as typo

This change fixes a typo in the adapting-existing doc.

Closes: #1850
Approved by: cgwalters

6 years agoprepare-root: remove ignored MS_MGC_VAL flag
Tristan Cacqueray [Sun, 5 May 2019 03:33:35 +0000 (03:33 +0000)]
prepare-root: remove ignored MS_MGC_VAL flag

Since Linux 2.4 this is no longer required and the flag is ignored.

Closes: #1851
Approved by: cgwalters

6 years agodocs: fix switchroot source link
Tristan Cacqueray [Sat, 4 May 2019 21:59:52 +0000 (21:59 +0000)]
docs: fix switchroot source link

This change updates the switchroot source relative url to an absolute url so
that it works from the readthedocs service. This change also removes
redundant switchroot.sh reference.

Closes: #1849
Approved by: cgwalters

6 years agoPost-release version bump
Jonathan Lebon [Wed, 24 Apr 2019 20:14:24 +0000 (16:14 -0400)]
Post-release version bump

Closes: #1844
Approved by: rfairley

6 years agoRelease 2019.2
Jonathan Lebon [Wed, 24 Apr 2019 20:12:31 +0000 (16:12 -0400)]
Release 2019.2

Also add `ostree_repo_get_bootloader` to the public API, which was
missed when it was initially merged.

Closes: #1844
Approved by: rfairley

6 years agoprepare-root: Fix compilation with --with-static-compiler
Colin Walters [Thu, 25 Apr 2019 09:08:23 +0000 (09:08 +0000)]
prepare-root: Fix compilation with --with-static-compiler

Closes: https://github.com/ostreedev/ostree/issues/1845
Closes: #1846
Approved by: akiernan

6 years agolib/deploy: Support locking deployment finalization
Jonathan Lebon [Tue, 16 Apr 2019 21:12:08 +0000 (17:12 -0400)]
lib/deploy: Support locking deployment finalization

Teach `ostree-finalize-staged.service` to check for a file in `/run` to
determine if it should do the finalization. This will be used in
RPM-OSTree, where we want to be able to separate out "preparing updates"
from "making update the default" for more fine-grained control. See:
https://github.com/projectatomic/rpm-ostree/issues/1748

Closes: #1841
Approved by: cgwalters

6 years agolib/deploy: Write to journal if nothing to finalize
Jonathan Lebon [Tue, 16 Apr 2019 21:08:24 +0000 (17:08 -0400)]
lib/deploy: Write to journal if nothing to finalize

This can happen if a deployment was staged and later cleaned up. Though
just as a helper when debugging issues, let's explicitly mention that
case.

Closes: #1841
Approved by: cgwalters

6 years agolib: Add ot_journal_* helper macros
Jonathan Lebon [Tue, 16 Apr 2019 21:06:01 +0000 (17:06 -0400)]
lib: Add ot_journal_* helper macros

Rather than wrapping each instance of `sd_journal_*` with
`HAVE_SYSTEMD`, let's just add some convenience macros that are just
no-op if we're not compiling with systemd.

Closes: #1841
Approved by: cgwalters

6 years agogrub2: Generate config from pending deployment
Robert Fairley [Thu, 14 Mar 2019 23:16:07 +0000 (19:16 -0400)]
grub2: Generate config from pending deployment

Generate a grub2 config using the pending deployment, if a grub2
bootloader is detected in the sysroot. Allows grub2-mkconfig
to run if there are no previous deployments.

Fixes: #1774
Closes: #1831
Approved by: jlebon

6 years agotests/test-admin-deploy-none: Split grub.cfg directory and file creation
Robert Fairley [Wed, 3 Apr 2019 16:25:04 +0000 (12:25 -0400)]
tests/test-admin-deploy-none: Split grub.cfg directory and file creation

Split the creation of the directory containing grub.cfg, and the creation
of the file, so that a failure in the mkdir command will fail the test
and not attempt the touch command.

Closes: #1831
Approved by: jlebon

6 years agotests/test-admin-deploy-none: Update comment for #1774 status
Robert Fairley [Thu, 14 Mar 2019 23:17:37 +0000 (19:17 -0400)]
tests/test-admin-deploy-none: Update comment for #1774 status

Closes: #1831
Approved by: jlebon

6 years agolib/sysroot: Match deployment /usr mode for overlay
Jonathan Lebon [Wed, 24 Apr 2019 13:42:56 +0000 (09:42 -0400)]
lib/sysroot: Match deployment /usr mode for overlay

Rather than hardcoding 0755, let's directly look at what `/usr`'s mode
is and copy it when creating the overlay.

Closes: #1843
Approved by: cgwalters

6 years agolib/sysroot: Call `chmod` on overlay dirs when unlocking
Jonathan Lebon [Tue, 23 Apr 2019 20:03:48 +0000 (16:03 -0400)]
lib/sysroot: Call `chmod` on overlay dirs when unlocking

Otherwise, we'll be subject to whatever `umask` is currently. Normally,
processes should respect `umask` when creating files and directories,
but specifically for `ostree admin unlock` (or `rpm-ostree usroverlay`),
this poses a problem since e.g. a `/usr` with mode 0700 will break any
daemon that doesn't run as root and needs to read files under `/usr`,
such as polkitd.

This patch just does a `chmod()` after the `mkdir()`. An alternative
would be to do `umask(0000)` after forking into the child process
that'll call `mount()`, but that'd require also moving the `mkdir()`
calls into there, making for a more intrusive patch.

Closes: #1843
Approved by: cgwalters

6 years agotests/test-config: Fix regex causing Debian test failure
Robert Fairley [Wed, 17 Apr 2019 16:03:55 +0000 (12:03 -0400)]
tests/test-config: Fix regex causing Debian test failure

Fix CI failure due to mismatching quotes in the error output given
between Fedora and Debian test suites.

Example of the error in Debian Stretch:
https://api.travis-ci.org/v3/job/519335717/log.txt

Closes: #1839
Approved by: cgwalters

6 years agotravis: Update to Debian Stretch
Robert Fairley [Fri, 12 Apr 2019 16:10:41 +0000 (12:10 -0400)]
travis: Update to Debian Stretch

Update to Stretch to use updated version of glib2.0. Fixes CI
failure when parsing error output in tests/test-config.sh. See:

https://github.com/ostreedev/ostree/pull/1838#issuecomment-482186680
https://api.travis-ci.org/v3/job/518830164/log.txt

Closes: #1839
Approved by: cgwalters

6 years agoostree-prepare-root: Log journal message after finding deployment
Jonathan Lebon [Tue, 25 Sep 2018 19:42:28 +0000 (15:42 -0400)]
ostree-prepare-root: Log journal message after finding deployment

Log a structured journal message when resolving the deployment path.
This will be used by the `rpm-ostree history` command to find past
deployments the system has booted into.

Closes: #1842
Approved by: cgwalters

6 years agolib/repo-refs: Allow resolving local collection-refs
Matthew Leeds [Thu, 21 Feb 2019 23:47:16 +0000 (15:47 -0800)]
lib/repo-refs: Allow resolving local collection-refs

Currently for a "normal" refspec you can choose to use
ostree_repo_resolve_rev_ext() instead of ostree_repo_resolve_rev() if
you only want to look at local refs (in refs/heads/) not remote ones.
This commit provides the analogous functionality for
ostree_repo_resolve_collection_ref() by adding a flag
OSTREE_REPO_RESOLVE_REV_EXT_LOCAL_ONLY and implementing it. This
will be used by Flatpak.

Closes: #1825
Approved by: jlebon

6 years agolib/repo-refs: Add a flag to exclude listing from refs/mirrors
Matthew Leeds [Thu, 21 Feb 2019 23:39:45 +0000 (15:39 -0800)]
lib/repo-refs: Add a flag to exclude listing from refs/mirrors

Currently the flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_REMOTES for
ostree_repo_list_collection_refs() means that refs in refs/remotes/
should be excluded but refs in refs/mirrors/ should still be checked, in
addition to refs/heads/ which is always checked. However in some
situations you want to exclude both remote and mirrored refs and only
check local "owned" ones. So this
commit adds a new flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_MIRRORS which
lets you exclude refs/mirrors/ from the listing.

This way we can avoid breaking API but still allow the listing of local
collection-refs.

The impetus for this change is that I'm changing Flatpak to make more
use of refs/mirrors, and we need a way to specify that a collection-ref
is local when using ostree_repo_resolve_collection_ref() in, for
example, the implementation of the repo command. The subsequent commit
will make the changes needed there.

Closes: #1825
Approved by: jlebon

6 years agolib/repo-refs: Fix resolving collection-refs
Matthew Leeds [Wed, 20 Feb 2019 21:18:31 +0000 (13:18 -0800)]
lib/repo-refs: Fix resolving collection-refs

My last commit "lib/repo-refs: Resolve collection-refs in-memory and in
parent repos" changed ostree_repo_resolve_collection_ref() to check the
in-memory set of refs *after* failing to find the ref on disk but that's
not what we want. We want to use the in-memory set of refs first,
because those are the most up to date commits, and then fall back to the
on-disk repo and finally fall back to checking any parent repo. This
commit makes such a change to the order of operations, which is
consistent with how ostree_repo_resolve_rev() works.

Aside from this change being logical, it also fixes some unit test
failures on an unmerged branch of flatpak:
https://github.com/flatpak/flatpak/pull/2705

Also, tweak the comments here.

Closes: #1825
Approved by: jlebon

6 years agoboot: Activate ostree-finalize-staged even earlier
Jonathan Lebon [Fri, 12 Apr 2019 17:49:08 +0000 (13:49 -0400)]
boot: Activate ostree-finalize-staged even earlier

Really, all `ostree admin finalize-staged` needs is access to `/sysroot`
and `/boot`. So let's activate it right after `local-fs.target` so that
it gets deactivated later in the shutdown process. This should allow us
to conflict with less services still running and possibly writing things
to `/etc`.

Related: https://bugzilla.redhat.com/show_bug.cgi?id=1672283

Closes: #1840
Approved by: cgwalters

6 years agotests: Put a regex in quotes
Matthew Leeds [Tue, 2 Apr 2019 02:04:05 +0000 (19:04 -0700)]
tests: Put a regex in quotes

Closes: #1834
Approved by: rfairley

6 years agotests: Escape periods when appropriate
Matthew Leeds [Tue, 2 Apr 2019 02:01:49 +0000 (19:01 -0700)]
tests: Escape periods when appropriate

Don't match any character in a regular expression when we only want to
match a period.

Closes: #1834
Approved by: rfairley

6 years agolib/repo: fix leak of `bootloader` field
Laurent Bonnans [Thu, 11 Apr 2019 10:04:52 +0000 (12:04 +0200)]
lib/repo: fix leak of `bootloader` field

Added in 21ebc7d21edd00c3fe0ce80ea68ba3f8f5e41ae3

Closes: #1837
Approved by: rfairley

7 years agolib: Check for http_proxy being the empty string
Matthew Leeds [Tue, 2 Apr 2019 20:40:11 +0000 (13:40 -0700)]
lib: Check for http_proxy being the empty string

On at least one user's computer, g_getenv("http_proxy") returns the
empty string, so check for that and treat it as no proxy rather than
printing a warning.

See https://github.com/flatpak/flatpak/issues/2790

Closes: #1835
Approved by: cgwalters

7 years agolib/repo-pull: Allow the keyring remote to be overridden
Matthew Leeds [Thu, 14 Feb 2019 08:54:09 +0000 (00:54 -0800)]
lib/repo-pull: Allow the keyring remote to be overridden

Currently the P2P code requires you to trust every remote you have
configured to the same extent, because a remote controlled by a
malicious actor can serve updates to refs (such as Flatpak apps)
installed from other remotes.[1] The way this attack would play out is
that the malicious remote would deploy the same collection ID as the
victim remote, and would then be able to serve updates for it.

One possible remedy would be to make it an error to configure remotes
such that two have the same collection ID but differing GPG keys. I
attempted to do that in Flatpak[2] but it proved difficult because it is
valid to configure two remotes with the same collection ID, and they may
then each want to update their keyrings which wouldn't happen
atomically.

Another potential solution I've considered is to add a `trusted-remotes`
option to ostree_repo_find_remotes_async() which would dictate which
keyring to use when pulling each ref. However the
ostree_repo_finder_resolve_async() API would still remain vulnerable,
and changing that would require rewriting a large chunk of libostree's
P2P support.

So this commit represents a third attempt at mitigating this security
hole, namely to have the client specify which remote to use for GPG
verification at pull time. This way the pull will fail if the commits
are signed with anything other than the keys we actually trust to serve
updates.

This is implemented as an option "ref-keyring-map" for
ostree_repo_pull_from_remotes_async() and
ostree_repo_pull_with_options() which dictates the remote to be used for
GPG verification of each collection-ref. I think specifying a keyring
remote for each ref is better than specifying a remote for each
OstreeRepoFinderResult, because there are some edge cases where a result
could serve updates to refs which were installed from more than one
remote.

The PR to make Flatpak use this new option is here[3].

[1] https://github.com/flatpak/flatpak/issues/1447
[2] https://github.com/flatpak/flatpak/pull/2601
[3] https://github.com/flatpak/flatpak/pull/2705

Closes: #1810
Approved by: cgwalters

7 years agostatic-delta: Change `show` to display from/to commits
Colin Walters [Sat, 16 Feb 2019 20:22:30 +0000 (20:22 +0000)]
static-delta: Change `show` to display from/to commits

When writing a delta to a file this may not always be recorded
in the filename, and it's useful data.

Ref: https://mail.gnome.org/archives/ostree-list/2019-February/msg00000.html

This also required teaching `show` to accept a file path.

Note...for some reason `test-deltas.sh` breaks when run from
a tty - we get `SIGTTIN` which implies something is reading from
the tty but it wasn't obvious to me what.

Closes: #1823
Approved by: jlebon

7 years agolib/deploy: Don't include version twice in bootmenu title
Jonathan Lebon [Fri, 1 Mar 2019 14:16:55 +0000 (09:16 -0500)]
lib/deploy: Don't include version twice in bootmenu title

In Silverblue right now, the boot menu title looks like this:

    Fedora 29.20190301.0 (Workstation Edition) 29.20190301.0 (ostree)

This is because RPM-OSTree's `mutate-os-release` feature is enabled,
which injects the OSTree version string directly into `VERSION` and
`PRETTY_NAME`. So appending the version string again is a bit redundant.
Let's just do a simple substring check here before adding the version to
the title.

Closes: #1829
Approved by: cgwalters

7 years agoAdd sysroot.bootloader repo config key
Robert Fairley [Thu, 14 Feb 2019 21:18:01 +0000 (16:18 -0500)]
Add sysroot.bootloader repo config key

The sysroot.bootloader key configures the bootloader
that OSTree uses when deploying a sysroot. Having this key
allows specifying behavior not to use the default bootloader
backend code, which is preferable when creating a first
deployment from the sysroot (#1774).

As of now, the key can take the values "auto" or "none". If
the key is not given, the value defaults to "auto".

"auto" causes _ostree_sysroot_query_bootloader() to be used
when writing a new deployment, which is the original behavior
that dynamically detects which bootloader to use.

"none" avoids querying the bootloader dynamically. The BLS
config fragments are still written to
sysroot/boot/loader/entries for use by higher-level software.

More values can be supported in future to specify a single
bootloader, different behavior for the bootloader code, or
a list of bootloaders to try.

Resolves: #1774

Closes: #1814
Approved by: jlebon

7 years agolibotutil: Add ot_keyfile_get_value_with_default_group_optional()
Robert Fairley [Wed, 20 Feb 2019 05:56:54 +0000 (00:56 -0500)]
libotutil: Add ot_keyfile_get_value_with_default_group_optional()

Add ot_keyfile_get_value_with_default_group_optional() which allows
getting values from keys where the group is optional in the config
file. This is preparatory to add the sysroot.bootloader repo config
key, where the sysroot group is optional.

Closes: #1814
Approved by: jlebon

7 years agolibotutil: Rename to ot_keyfile_get_string_list_with_separator_choice()
Robert Fairley [Thu, 14 Feb 2019 21:14:03 +0000 (16:14 -0500)]
libotutil: Rename to ot_keyfile_get_string_list_with_separator_choice()

Rename ot_keyfile_get_string_as_list() to
ot_keyfile_get_string_list_with_separator_choice() which expresses
more clearly why the function is needed. Also shorten the
function comment.

Closes: #1814
Approved by: jlebon

7 years agotests/test-config: Remove extra space
Robert Fairley [Fri, 1 Mar 2019 20:24:07 +0000 (15:24 -0500)]
tests/test-config: Remove extra space

Closes: #1814
Approved by: jlebon

7 years agoostree-grub-generator: Comment wording fix
Robert Fairley [Fri, 8 Feb 2019 22:28:56 +0000 (17:28 -0500)]
ostree-grub-generator: Comment wording fix

Closes: #1814
Approved by: jlebon

7 years agoci/rpmostree: Bump to 2019.1
Colin Walters [Thu, 24 Jan 2019 10:37:46 +0000 (11:37 +0100)]
ci/rpmostree: Bump to 2019.1

Should fix the issue that CI broke due to libsolv not being sync'd.

Closes: #1806
Approved by: jlebon

7 years agoci/rpmostree: Fix use of `fatal`
Colin Walters [Thu, 24 Jan 2019 10:36:40 +0000 (11:36 +0100)]
ci/rpmostree: Fix use of `fatal`

We're not using libtest, just inline it.

Closes: #1806
Approved by: jlebon

7 years agoostree/config: Check for too many args
Matthew Leeds [Fri, 1 Mar 2019 00:19:41 +0000 (16:19 -0800)]
ostree/config: Check for too many args

Currently it's not an error to provide too many arguments to an ostree
config command. Change it so we print usage information in that case,
and update the unit tests.

Closes: #1743
Approved by: cgwalters

7 years agoman/ostree-config: Consistently use GROUPNAME placeholder
Matthew Leeds [Thu, 28 Feb 2019 23:15:51 +0000 (15:15 -0800)]
man/ostree-config: Consistently use GROUPNAME placeholder

It doesn't make much sense to use SECTIONNAME in some places and
GROUPNAME in others when they mean the same thing.

Closes: #1743
Approved by: cgwalters

7 years agoostree/config: Use g_autoptr instead of goto
Matthew Leeds [Fri, 28 Sep 2018 22:45:43 +0000 (15:45 -0700)]
ostree/config: Use g_autoptr instead of goto

It seems cleaner to make the GKeyFile a g_autoptr variable and just
return rather than using the "goto out;" idiom.

Closes: #1743
Approved by: cgwalters

7 years agoostree/config: Add an "unset" operation
Matthew Leeds [Fri, 28 Sep 2018 22:36:49 +0000 (15:36 -0700)]
ostree/config: Add an "unset" operation

Currently there's a way to set a key to the empty string but there's no
way to unset it completely (remove the key from the group). This might
be helpful for instance if you want to temporarily set
"core.lock-timeout-secs" to a specific value for the duration of one
operation and then return it to the default after that operation
completes.

This commit implements an "unset" operation for the config command, adds
a unit test, and updates the man page.

Closes: #1743
Approved by: cgwalters

7 years agolib/repo-refs: Resolve collection-refs in-memory and in parent repos
Matthew Leeds [Thu, 14 Feb 2019 09:33:20 +0000 (01:33 -0800)]
lib/repo-refs: Resolve collection-refs in-memory and in parent repos

Currently the behavior of ostree_repo_resolve_rev() is that it tries to
resolve a ref to a commit by checking the refs/ directories, but also by
checking for in-memory ref-checksum pairs which are part of an
in-progress transaction and also by checking the parent repo if one
exists. Currently ostree_repo_resolve_collection_ref() only checks the
refs/ directories, so this commit makes its behavior analagous since it
is the analagous API which supports collection-refs.

The impetus for this was that currently Flatpak uses
ostree_repo_resolve_rev() to load a commit after doing a P2P pull in
flatpak_dir_do_resolve_p2p_refs(), but that assumes the ref came from
the same remote that originally provided it, which might not be the case
if more than one remote has the same collection ID configured. And
changing Flatpak to use ostree_repo_resolve_collection_ref() doesn't
work without this patch.

Closes: #1821
Approved by: pwithnall

7 years agoman: Document ostree --version
Matthew Leeds [Thu, 14 Feb 2019 19:08:28 +0000 (11:08 -0800)]
man: Document ostree --version

Closes: #1820
Approved by: rfairley

7 years agotests: Remove unused includes in libostreetest.c
Matthew Leeds [Sat, 9 Feb 2019 00:01:54 +0000 (16:01 -0800)]
tests: Remove unused includes in libostreetest.c

Since commit a06bd82cd we no longer use OVERLAYFS_SUPER_MAGIC or
statfs() so remove the includes for linux/magic.h and sys/vfs.h

Closes: #1815
Approved by: cgwalters

7 years agotests: Update for glib 2.59.2
Jeremy Bicha [Mon, 11 Feb 2019 01:54:05 +0000 (20:54 -0500)]
tests: Update for glib 2.59.2

glib 2.59.2 uses a non-breaking space instead of a space to
separate the quantity and unit in g_format_size() so update
our test to handle both a plain space and a non-breaking space.

See https://gitlab.gnome.org/GNOME/glib/issues/1625

Closes: #1818
Approved by: cgwalters

7 years agotests: Don't skip collection-based summary test
Matthew Leeds [Wed, 23 Jan 2019 00:50:32 +0000 (16:50 -0800)]
tests: Don't skip collection-based summary test

OSTree's P2P support used to be a compile time option but is now always
enabled. One of the tests was still checking for the old feature flag,
"experimental", which has been renamed to "p2p" and skipping for that
reason. Delete the check so the test always runs.

Closes: #1804
Approved by: pwithnall

7 years agolib/repo: Fix Since: annotation for REMOTE_CHANGE_REPLACE
Matthew Leeds [Fri, 8 Feb 2019 21:52:52 +0000 (13:52 -0800)]
lib/repo: Fix Since: annotation for REMOTE_CHANGE_REPLACE

Closes: #1813
Approved by: jlebon

7 years agoremote-add: Add --force option to add or replace remote
Dan Nicholson [Tue, 12 Sep 2017 22:05:08 +0000 (17:05 -0500)]
remote-add: Add --force option to add or replace remote

This uses the OSTREE_REPO_REMOTE_CHANGE_REPLACE operation to add a
remote or replace an existing one. This is roughly the opposite of
--if-not-exists and will raise an error if both options are passed.

Closes: #1166
Approved by: cgwalters

7 years agoman: Document remote add --if-not-exists option
Dan Nicholson [Tue, 12 Sep 2017 20:20:46 +0000 (15:20 -0500)]
man: Document remote add --if-not-exists option

Closes: #1166
Approved by: cgwalters

7 years agorepo: Add OstreeRepoRemoteChange replace operation
Dan Nicholson [Tue, 12 Sep 2017 17:23:31 +0000 (12:23 -0500)]
repo: Add OstreeRepoRemoteChange replace operation

Add the OSTREE_REPO_REMOTE_CHANGE_REPLACE operation to the
OstreeRepoRemoteChange enum. This operation will add a remote or replace
an existing one. It respects the location of the remote configuration
file when replacing and the remotes config dir settings when adding a
new remote.

Closes: #1166
Approved by: cgwalters

7 years agolib/repo-pull: Fix docs for n-network-retries option
Matthew Leeds [Wed, 6 Feb 2019 22:50:13 +0000 (14:50 -0800)]
lib/repo-pull: Fix docs for n-network-retries option

Add "Since: 2018.6" and copy the docs to
ostree_repo_pull_from_remotes_async() which also supports it.

Closes: #1812
Approved by: jlebon

7 years agolib/commit: Don't set user.ostreemeta for bare → bare-user-only imports
Colin Walters [Wed, 16 Jan 2019 13:45:34 +0000 (13:45 +0000)]
lib/commit: Don't set user.ostreemeta for bare → bare-user-only imports

The point of `bare-user-only` is it doesn't have xattrs; it's a
special case in import handling.

Closes: #1803
Approved by: jlebon

7 years agolib/commit: Set xattr on correct fd for bare-user → bare-user imports
Colin Walters [Wed, 16 Jan 2019 13:36:55 +0000 (08:36 -0500)]
lib/commit: Set xattr on correct fd for bare-user → bare-user imports

Another fix to https://github.com/ostreedev/ostree/pull/1771

Closes: #1803
Approved by: jlebon

7 years agolib/commit: Fix bare → bare imports
Colin Walters [Wed, 16 Jan 2019 13:30:53 +0000 (08:30 -0500)]
lib/commit: Fix bare → bare imports

Regression from https://github.com/ostreedev/ostree/pull/1771

This broke rpmostreepayload in Anaconda where we import a bare repo:
https://openqa.fedoraproject.org/tests/345339#step/_do_install_and_reboot/4

Reported-by: Adam Williamson <adamwill@fedoraproject.org>
Closes: #1803
Approved by: jlebon

7 years agoREADME: add note about Liri OS
Micah Abbott [Tue, 15 Jan 2019 14:45:02 +0000 (09:45 -0500)]
README: add note about Liri OS

Closes: #1802
Approved by: rfairley

7 years agoPost-release bump
Colin Walters [Fri, 11 Jan 2019 15:46:36 +0000 (10:46 -0500)]
Post-release bump

Closes: #1800
Approved by: jlebon

7 years agoRelease 2019.1
Colin Walters [Fri, 11 Jan 2019 15:46:19 +0000 (10:46 -0500)]
Release 2019.1

Closes: #1800
Approved by: jlebon

7 years agopull: If built with --disable-http2, allow enabling via http2=1
Colin Walters [Tue, 8 Jan 2019 14:41:26 +0000 (09:41 -0500)]
pull: If built with --disable-http2, allow enabling via http2=1

We have a `http2=[0|1]` remote config option; let's have the
`--disable-http2` build option define the default for that.  This way
it's easy to still enable http2 for testing even if
we have it disabled by default.

Closes: #1798
Approved by: jlebon

7 years agofixup! admin/status: test GPG signature check
Colin Walters [Tue, 8 Jan 2019 15:24:10 +0000 (15:24 +0000)]
fixup! admin/status: test GPG signature check

7 years agoadmin/status: test GPG signature check
Rafael Fonseca [Mon, 7 Jan 2019 14:12:42 +0000 (15:12 +0100)]
admin/status: test GPG signature check

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
Closes: #1797
Approved by: cgwalters

7 years agoatomic-upgrades doc: fix example url to fetch a ref checksum
Sinny Kumari [Wed, 2 Jan 2019 10:33:19 +0000 (16:03 +0530)]
atomic-upgrades doc: fix example url to fetch a ref checksum

Signed-off-by: Sinny Kumari <sinny@redhat.com>
Closes: #1795
Approved by: cgwalters